Adaptive Finite-Difference Interval Estimation for Noisy Derivative-Free Optimization

A common approach for minimizing a smooth nonlinear function is to employ finite-difference approximations the gradient. While this can be easily performed when no error present within evaluations, noisy, optimal choice requires information about noise level and higher-order derivatives of function, which often unavailable. Given we propose bisection search finding interval any scheme that balances truncation error, arises from in Taylor series approximation, measurement results evaluation. Our procedure produces reliable estimates at low cost without explicitly approximating derivatives. We show its numerical reliability accuracy on set test problems. When combined with limited memory BFGS, obtain robust method noisy black-box functions, as illustrated subset unconstrained CUTEst problems synthetically added noise.